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MICROSCOPY IMAGING APPARATUS AND MEHOD FOR GENERATING AN IMAGE 

The present invention relates to a microscopy imaging 
5 apparatus and a method for generating an image, and in 
particular an apparatus and method for generating an 
optically sectioned image. 

With a conventional wide-field microscope it is not 
possible to obtain a proper two-dimensional optically 
id sectioned image of a three dimensional specimen because the 
resultant image contains contributions from out-of-focus 
specimen structures above and below the specimen object 
plane which blur the image of the specimen object plane. 
Conventional wide-field microscopes are unable to reject 
is these out-of -focus details. 

To overcome this problem confocal microscopes have 
been developed. These devices use a point light source 
located in a plane which. is optically conjugate to the 
specimen object plane. This arrangement allows out -of - 
20 focus light to be rejected, but the illuminating light beam 
has to be laterally scanned to build up the optically 
sectioned image point -by-point . This leads to long 
acquisition times, particularly if it is desired to "stack" 
optically sectioned images obtained at successive focal 
25 positions to form a three dimensional image of the 

specimen. Also the light efficiency of such devices tends 
to be low. 

In order to avoid scanning and to reduce acquisition 
times a number of approaches have been suggested. One of 
30 these involves the use of a Nipkow disk in which the 

pinhole is replaced by a mult i- aperture rotating disk that 
enables parallel observation of several confocal volumes. 
Even so, only about 2% of the total illumination light 
budget is used for imaging. 
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In an attempt to overcome this problem, Verveer et al . 
(Verveer P.J., Gemkow M.J. and Jovin M. 7 J*. Microscopy, 
Vol. 189(3), (1998), 192-198), proposed replacing the 
Nipkow disk with a digital micro -mirror device which serves 
as a programmable pinhole array. The image thus -obtained 
consists of superimposed confocal and conventional images. 
The pure confocal contribution is recoverable by 
subtracting the conventional image, but at the cost of 
increasing noise. 

WO 98/4574 5 discloses an alternative approach in which 
an object is illuminated by a periodic pattern and at least 
three specimen images are recorded at different spatial 
phases of the pattern. The three images are then combined 
1 by image processing which removes the periodic pattern and 
out-of -focus contributions, and thereby produces an 
optically sectioned image. However, this approach is not 
convenient for implementing real-time imaging. 

An object of the present invention is to provide a 
method of generating an optically sectioned specimen image 
which, like conventional confocal techniques, comprises 
substantially only in- focus detail, but has a more 
favourable light budget and acquisition time. A further 
object is to provide a method of generating an optically 
sectioned specimen image which facilitates real-time 
imaging . 

In a first aspect, the present invention provides a 
method of generating an optically sectioned image of a 
specimen comprising the steps of: illuminating .the specimen 
with a modulating, spatially periodic illumination pattern; 
imaging said specimen on a conjugate image plane; acquiring 
a plurality of signals at respective positions on said 
image plane, each signal (which is typically an electrical 
signal) corresponding to the incident light intensity at 
that position and having an oscillatory component caused by 
the modulation of - the illumination pattern; and measuring a 
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characteristic (typically the amplitude) of the oscillatory 
component of each of the signals, whereby the measured 
characteristics when combined in their relative positions 
generate an optically sectioned image of the specimen. 

Thus, the present invention is based on the 
realisation that analysis of the temporal variation of the 
light from the specimen allows out-of-focus contributions 
to be rejected- Such analysis is amenable to automated 
processing, which opens the door to real-time imaging. 

This is in contrast to the approach of WO 98/45745 
where discrete images rather than temporal variations are 
analysed. 

Preferably the method further comprises a step, 
between the acquisition and measuring steps, of filtering 
each acquired signal to isolate the oscillatory component 
therefrom . 

Typically, the optically sectioned image will be 
pixellated, the value (e.g. brightness or colour value) of 
each image pixel being determined by the measured 
characteristic of the respective oscillatory component, and 
the position of each pixel corresponding to a respective 
position on the image plane. 

The method may be repeated for different focal 
positions to produce a series of optically sectioned images 
of the specimen. These can then be combined to form a 
three-dimensional image of the specimen. 

Preferably, the spatially periodic illumination 
pattern is modulated to produce a predetermined modulation 
frequency at respective specimen positions, whereby the 
oscillatory components of the signals acquired at 
corresponding positions on the image plane have 
substantially the same frequency. By the' "modulation 
frequency" we mean the reciprocal of the time required for 
a position on- the specimen to undergo one complete 
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illumination cycle from light to dark and back to light . 
again. 

In preferred embodiments, the illumination pattern has 
one -dimensional periodicity. For example the pattern may 
be a fringe pattern e.g. comprising parallel linear fringes 
or concentric circular fringes. Such patterns have well- 
defined spatial periodicities and may be produced e.g. by 
interferometry or by an illuminated mask. They can also be 
modulated by moving or shifting the elements of the 
patterns in such a way to ensure that substantially all 
imaged specimen positions experience substantially equal 
modulation frequencies and time -integrated illumination 
intensities. 

For example, a pattern of parallel linear fringes may 
be moved by shifting the fringes across the specimen in the 
direction lateral to the fringes, the modulation frequency 
is then the inverse of the time taken to shift the fringes 
by one fringe spacing. A pattern of concentric circular 
fringes may be moved by expanding the fringes away from or 
contracting them towards the common centre of the fringes, 
the modulation frequency being the inverse of the time 
taken for a fringe to grow (or shrink) to the size of the 
ad j acent fringe . 

Preferably, the illumination pattern is modulated to 
produce a modulation frequency of at least 100 Hz (more 
preferably of at least 1 kHz, 10 kHz or 100 kHz) . 

Typically the incident' light at the image plane 
comprises reflected or transmitted light, and/or light 
emitted by the specimen (such as fluorescent light) in 
response to the illumination pattern. 

A further aspect of the present invention provides a 
method of processing image data to generate an optically 
sectioned image comprising: (a) receiving data which 
comprises a plurality of signals previously acquired by 
performing the steps of (i) illuminating a specimen with a 
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modulating, spatially periodic illumination pattern, (ii) 
imaging said specimen on a conjugate image plane, and (iii) 
acquiring a plurality of signals at respective positions on 
said image plane, each signal corresponding to the incident 
light intensity at that position and having an oscillatory 
component caused by the modulation of the illumination 
pattern ; and ( b ) 

measuring a characteristic (typically the amplitude) of the 
oscillatory component of each of the signals, whereby the 
measured characteristics when combined in their relative 
positions generate an optically sectioned image of the 
specimen. 

Thus this aspect of the invention relates to the 
processing of image data previously acquired by performing 
the illumination, imaging and acquisition steps of the 
previous aspect. Optional and preferred features of the 
previous aspect apply also to this aspect. 

For example, the method may further comprise 
a step, between the receiving and measuring steps, of 
filtering each acquired signal to isolate the oscillatory 
component therefrom * 

A further aspect of the invention provides a 
microscopy imaging apparatus comprising: illumination means 
for illuminating a specimen with a modulating, spatially 
periodic illumination pattern; imaging means for imaging 
said specimen on a conjugate image plane; acquisition means 
for acquiring a plurality of signals at respective 
positions on said image plane, each signal corresponding to 
the incident light intensity at that position and having an 
oscillatory component caused by the modulating illumination 
pattern; and processor means for measuring a characteristic 
(typically the amplitude) of the oscillatory component of 
each of the signals, whereby the measured characteristics 
when combined in their relative positions generate an 
optically sectioned image of the specimen. 
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Preferably the processor , means also filters each 
acquired signal to isolate the oscillatory component 
therefrom prior to measuring the characteristic of the 
oscillatory component. 

In one embodiment, the illumination means comprises 
means for generating a spatially periodic interference 
illumination pattern, such as may be generated by two 
overlapping coherent beams of light. The coherent beams of 
light may be produced by a laser or lasers. 

Alternatively the illumination means may comprise a 
light source which illuminates the specimen through a 
patterned grid, the grid pattern corresponding to the 
illumination pattern. Such a source may be incoherent. 
Furthermore, it may illuminate the specimen with. white 
light. 

The acquisition means may comprise an array of light 
detectors for respectively detecting light intensity at the 
plurality of image plane positions. Typically such 
detectors operate in parallel. Suitable one- and two- . 
dimensional signal detector arrays are described in EP-A- 
1065809. If a one-dimensional array is used, generation of 
a two-dimensional optically sectioned image requires the 
array to be scanned across the image plane. Preferably, 
however, a two-dimensional array is used so that light 
intensities across substantially the whole image plane are 
detected simultaneously. This reduces the. image 
acquisition time. 

With a two-dimensional array of detectors, each 
detector may acquire a signal corresponding to one pixel of 
a pixellated optically sectioned image. With a one- 
dimensional scanned array of detectors, each detector, may 
acquire signals corresponding to a line of pixels of a 
^pixellated optically sectioned image 

Preferably, the processor means comprises a plurality 
of signal processors for respectively measuring (and 
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optionally filtering) the oscillatory components of the 
acquired signals. Typically the processors operate in 
parallel to reduce the image generation time. 

More preferably, with an array of signal detectors, 
each signal detector has a dedicated signal processor, e.g. 
as described in EP-A-1065809 . Thus signal acquisition, 
filtering and subsequent measurement can be fully 
automated. 

Typically the illumination means modulates the 
illumination pattern to produce a predetermined 
illumination modulation frequency (e.g. of at least 100 Hz, 
1 kHz,' 10 kHz or 100 kHz) . Preferably the processor means 
is adapted to filter the acquired signals at substantially 
the same frequency in order to isolate effectively the 
oscillatory components from the signals. 

A further aspect of the invention provides a 
conversion kit for converting a conventional microscope 
into the microscopy imaging apparatus of the previous 
aspect, the conversion kit comprising: illumination means 
for illuminating a specimen with a modulating, spatially 
periodic illumination pattern; acquisition means for 
acquiring a plurality of signals at respective positions on 
a conjugate image plane onto which the microscope images 
the specimen, each signal corresponding to the incident 
light intensity at that position and having an oscillatory 
component caused by the modulating illumination pattern, 
and processor means for measuring a characteristic 
(typically the amplitude) of the oscillatory component of 
each of the signals, whereby the measured characteristics 
when combined in their relative positions generate an 
optically sectioned image of the specimen. 

Preferably the processor means filters each acquired 
signal to isolate the oscillatory component therefrom prior 
to measuring the characteristic of the oscillatory 
component . 
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The kit may have other preferred or optional features 
corresponding to preferred or optional features of the 
microscopy imaging apparatus of the previous aspect. 

Specific embodiments of the present invention will now 
be described with reference to the accompanying drawings in 
which : 

Fig. 1 shows the absolute value of the optical 
transfer function OTF as a function' of the generalised 
distance to the focal plane u = 4nw / A and the generalised 
spatial frequency v = A / (n sin a.f) , 

Fig. 2 is a flow chart which shows schematically the 
signal detection and processing steps performed by a 
detector/processor pair of an amplitude demodulation 
detector array (ADDA) , 

Fig. 3 shows a typical detector /processor pair 
frequency response, 

Fig. 4 shows a microscopy imaging apparatus according 
to a first embodiment of the present invention/ and 

Fig. 5 shows a microscopy imaging apparatus according 
to a second embodiment of the present invention. 

A spatially periodic illumination pattern comprising 
linear parallel fringes may be formed e.g. by interference 
between two mutually coherent light beams. The 
interference fringes are formed by planes of uniform 
illumination intensity extending essentially parallel to 
the optical axis. Thus on any plane perpendicular to the 
optical axis (such as the focal plane) the fringes appear 
as an interference pattern having a period given by the 
magnitude of the wave shift vector, AK = k x - k 2 , resolved 

onto that plane, where k L and k 2 being the k-vectors of the 
two laser beams. 

The resulting specimen illumination intensity, I eX / is 
given by: 

I ex (r, cp) = 1 + cos (2AKf + cp) , 
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where f is a position vector in a plane perpendicular to 
the optical axis, and cp is the phase lag produced when the 
fringes are modulated by being shifted in a direction 
lateral to the fringes. The value of cp is thus a function 
of time, t. For. example, if f is the illumination 
modulation frequency, the fringes are moved laterally by a 
whole fringe period in a time 1/f, so cp = 2ntf . 

The illumination pattern causes (e.g. reflected, 
transmitted or fluorescent) light to originate from the 
specimen, the light having an intensity which is 
proportionate to I ex and also to the specimen three- 
dimensional optical object structure, o(f ) . Thus the image 
intensity, Ii m , can be written as: 

I im(^/ <P> = offJI^r, cp) * p(f ) , 
where p(r) is the incoherent point spread function and * 
denotes a convolution . 

In k-space this can be written as: 

I im (k, cp) = 0(k)P(k) + - [e iq, 0(k + AK) 4- e" i<p O(k - AK)]p(k) , 

2 

where k is reciprocal distance, P(k) is the optical 
transfer function (OTF) , and O is the image frequency 
spectrum. The first term, 0(k)P(k) , corresponds to an image 
obtained using a conventional microscope, while the second 
term contains the image frequency spectrum shifted towards 
higher spatial frequencies. Only the second term is 
dependent on the phase lag, 9, of the fringe system. 
Effectively, the first term may be thought of -as a DC 
component and the second term may be thought of as an AC or 
oscillatory component. 

In the second term, the frequency shifted image 
spectrum is multiplied by the OTF. Fig. l shows the 
absolute value of the OTF according to an approximation 
given by Stokseth (A. Stokseth, Journal of the Optical 
Society of America, Vol. 59(10), (1969), 1314-1321) as a 
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function of the generalised distance to the focal plane 
u =b 4nw / X and the generalised spatial frequency 
v = X / (n sin a.f) , where w is the optical path length, X is 
the optical wavelength, nsina is the numerical aperture and 
f is the spatial frequency. Out -of -focus contributions 
(i.el contributions originating from specimen positions 
distant from the focal plane) are attenuated much more 
heavily than in-focus contributions (i.e. when u = 0) . 

The theoretical model discussed above is based on the 
well-known approach of describing imaging in terms of 
reciprocal k-space. For simplicity we have used the 
example of coherent illumination. However incoherent 
illumination can be adopted in a straightforward manner 
instead. For example, the linear parallel fringes may be 
formed by illuminating an appropriate mask and imaging the 
mask onto a focal plane coincident with the specimen. 

Consistent with the theoretical model, we can also 
describe the optical sectioning effect achieved by the 
present invention by a simple qualitative model in real x- 
y-z space, where the optical axis corresponds to the z- 
axis. Effectively, the moving periodic illumination 
pattern is only "visible" (in the sense of having strong 
contrast) in the imaged specimen within an optically 
sectioned volume which extends across an x-y plane. The 
visibility of the pattern drops off sharply along the 
optical axis in other x-y planes to either side of the 
optically sectioned volume. That is, away from the 
optically sectioned volume the pattern washes out and 
eventually forms non-periodic background illumination which 
contributes to the above-mentioned DC component. 

Thus moving out of focus means loosing fringe contrast 
and movement . Hence only those parts of the specimen 
within the optically sectioned volume modify (e.g. by 
reflection, transmission or fluorescence) the illuminating 
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light in such a way as to result in a light signals which 
have oscillatory components. These can be measured as AC 
signals which can then be used to construct an optically 
sectioned image . 

Thus optical sectioning can be achieved by locating an 
array of light detectors on the conjugate image plane and 
splitting the signal acquired at each detector into a DC 
and an AC component . The DC component corresponds to the 
time -invariant conventional microscope image and can be 
ignored, whereas the AC signal carries substantially only 
in- focus spatial frequency components and can be used to 
generate an optically sectioned image. 

A suitable detector array is the amplitude 
demodulation detector array (ADDA) described in EP-A- 
1065809. The ADDA is formed on a silicon chip as a two- 
dimensional array of photo sensor light detectors and 
corresponding integrated circuit signal processors. Each 
signal/processor pair corresponds to one pixel of the final 
optically sectioned image whereby effectively instantaneous 
acquisition of a complete image can be achieved. 

An advantage of using ADDAs is that all the signal 
processing functions can be performed by integrated 
circuitry on the silicon chips. ADDAs also have low 
sensitivity to power fluctuations. 

Fig. 2 is a flow chart which shows schematically the 
signal acquisition and processing steps performed by each 
detector/processor pair of an ADDA. . The photosensor 1 
captures the light and generates a corresponding electrical 
signal. The amplifier 2 together with the low-pass filter 
3 and current source 4 provide a feedback loop which 
results in the rejection of the DC component of the 
electrical signal and amplification of the AC component. 
The AC component is then rectified at rectifier 5 and a 
final stage low-pass filter 6 further filters and smoothes 
the AC component. 
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The analog output of the detector/processor pair is 
therefore effectively a measurement of the amplitude of the 
AC component/ and can then be sent to an analog/digital 
converter for combination with the outputs from the other 
pairs into the optically sectioned image. The image may be 
displayed, for example, on a video monitor ' (with the AC . 
component amplitude measurements converted into e.g. pixel 
brightness levels) or stored on a computer-readable memory 
device (such as a hard disc, RAM, CD etc.) . 

Fig. 3 shows a typical detector/processor pair 
frequency response. The response provides DC signal 
suppression and an AC signal pass band which has a maximum 
frequency response at about 200 kHz. For optimum 
compatibility with an ADDA array formed of such pairs, the. 
illumination pattern should be modulated so that the 
illumination modulation frequency is 200 kHz. 

An advantage of such a frequency response is that 
image noise contributions for the conventional microscope 
image are also suppressed. 

Fig. 4 shows a microscopy imaging apparatus according, 
to a first embodiment of the present invention. Light is 
produced by a coherent light -source 11 such as a laser, and 
is optionally attenuated by an optional attenuator 12. 
This light is split into two parallel beams at beam 
separator 13 and mirror or prism 14. Next, phase or 
frequency shifting elements 15 and 16 respectively operate 
on the beams. The two shifted beams are then recombined by 
mirrors 17 and 18 at the specimen 19 to form a continuously 
moving periodic illumination pattern which provides a 
uniform illumination modulation frequency at each point on 
the specimen 19. 

The light from the specimen is then imaged through 
lenses 2 0 and 21 (which may be for example a conventional 
wide field microscope observation stage arranged in 
transmission or reflection configuration) onto a two- 
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dimensional ADDA array 22, each detector/processor pair of 
the ADDA array having a maximum response frequency tuned to 
the illumination modulation frequency. 

Fig. 5 shows a microscopy imaging apparatus according 
5 to a second embodiment of the present invention. In this 
embodiment, light is produced by an incoherent light source 
31, before being optionally attenuated by optional 
attenuator 32. A periodic illumination pattern is 
generated by a mask 3 6 contained in housing 35 and located 

10 at an intermediate image position conjugated to the object. 
The pattern is imaged onto specimen 3 9 via the objective 
lens 40 of a conventional microscope observation optics 41. 
The mask is moved laterally by a piezo-element (not shown) 
to shift the image of the illumination pattern relative to 

15 the specimen. The observation optics then form an image of 
the specimen from scattered or emitted light at a two- 
dimensional ADDA array 42 located at a conjugate image 
plane . 



